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(54) [Title of the Invention] Data Embedding Method and Device 
(57) [Abstract] 

[Purpose] The present invention, which relates to devices which 
prevent the illicit copying of digitized video software - as typified by 
electronic movies, has as an object the provision of a method and 
device for embedding data in a video signal with almost no 
degradation in the video signal. 

[Constitution] A characteristic region detection device 1 detects 
regions in the input signal which satisfy designated conditions, and 
sends the detection signal to an interpolation processing device 2 and 
a temporary storage device 4. m the interpolation processing device 2, 
interpolation signals for characteristic regions are generated and sent 
to a selector 5. Meanwhile, embedding data is error-correction 
encoded by an error-correction encoding device 3, and is stored in the 
temporary storage device 4. In the temporar y storage device 4, each 
time that a detection signal is inputted from the characteristic region 
detection device 1, embedding data is outputted to the selector's 
controller, and based on this signal, in the selector 5, either the input 
signal or the signal from the interpolation processing device 2 is 
selected as output 



Input Signal 




Output Signal 



Embedding Data 



[Claims] 

[Claim 1] A data embedding method, characterized by detecting, 
as characteristic regions, elevated points or regions which have a 
value higher than a set value compared to the neighboring signal 
values, or sunken points or regions which have a value lower 
than a set value compared to the neighboring signal values, and 
embedding data in the video signal by cither swapping out said 
characteristic regions for another signal, or leaving [said 
characteristic regions] as is without any processing. 
[Claim 2] The data embedding method according to Claim 1, 
characterized by selecting a subset of the detected characteristic 
regions at random, and the repeatedly embedding the embedding 
data in the video signal. 

[Claim 3] A data embedding method, characterized by detecting, 
as characteristic regions, elevated points or regions which have a 
value which rise beyond a set value relative toboth ends on the 
respective lines of a video signal, or sunken points or regions 
which have a value lower than a set value relative to both ends 
on the respective lines of a video signal, and embedding data in 
the video signal by either taking out said characteristic regions 
and interpolating using the neighboring signal, or leaving [said 
characteristic regions] without any processing. 
[Claim 4] The data embedding method according to Claim 3, 
characterized by detecting, as characteristic regions, regions 
which, among line groups in the video signal which are included 
in the display of an object whose signal values differ from the 
background by more than a set value^howing the edge portions 
of the object 

[Claim 5] A data embedding device, characterized by being 
equipped with detection means for detecting, as characteristic 
regions, elevated points or regions which have a value higher 
than a set value compared to the neighboring signal values, or 
sunken points or regions which have a value lower than a set 
value compared to the neighboring signal values, and a means 
for converting said characteristic regions into other signals; and 
by possessing a means for selecting, as the output signal, either a 
signal in which the characteristic points or characteristic regions 
have been converted by said conversion means based on the 
embedding data, or a signal which has been left as is without 
any processing. 

[Claim 6] A data embedding device equipped with detection 
means for taking the difference between neighboring pixels in 
the scanning direction of the lines of a video signal, and 
detecting regions for which, observing in the scanning direction, 
this difference signal is larger than a set value in the positive 
direction, and the difference signal following this is larger than a 
set value in the negative direction, or the value which follows is 
lower than a set value and men the difference signal is larger 
than a set value in (he negative direction, or regions for which, 
observing in the scanning direction, this difference signal is 
larger than a set value in the negative direction, and the 
difference signal following this is larger than a set value in the 
positive direction, or the value which follows is lower than a set 
value and then the difference signal is larger than a set value in 
the positive direction; and possessing means for removing said 
detected characteristic regions and interpolating using the signal 
of the neighboring pixels, and means for selecting, as the output 
signal, either a signal based on said interpolation processing 
based on the embedded data, or a signal which has not been 
processed. 

[Claim 7] A data embedding device according to Claim 6, 
characterized by possessing detection means wherein, if 
characteristic regions on the respective lines have a portion in 
which the characteristic regions on adjacent lines overlap at a 
position in the horizontal direction, and these characteristic 



regions form a single continuous characteristic region, then from 
among these only the characteristic region in which bom edges 
are included on the line will be detected as a characteristic 
region. 

[Claim 8] A data embedding device according to Claim 6, 
characterized by possessing detection means wherein, if 
characteristic regions on the respective lines have a portion in 
which the characteristic regions on adjacent lines overlap at a 
position in the horizontal direction, and these characteristic 
regions form a single continuous characteristic region, then from 
among these only the region for which the size of the line is the 
smallest will be detected as a characteristic region. 
[Detailed Description of the Invention] 
[0001] 

[Field of Industrial Application] The present invention relates to 
data embedding devices which embed specific data into video 
software which should protect copyright - as typified by 
electronic movies - in advance such mat, when this video 
software is illicitly copied, it can be ascertained from whose 
software it was copied. 
[0002] 

[Prior Art] m terms of methods of preventing the illicit copying 
of video software, there have been methods which preclude 
illicit copiers by adding a signal which makes copying to a 
videotape recorder difficult, or by adding individualized data to 
the software itself to make it possible to identify the owner of 
the software that allowed the illicit copying. FIG. 6 is a data 
embedding method in a conventional copy guard. Separate from 
the signal which plays the actual video, the embedded signal is 
one wherein data for copyright protection, such as a copy 
prohibit/allow code, is embedded along with the pertinent data 
for playing of the video. Specifically, these were methods which 
on the playback side would read the pertinent [video] data, and 
at the same time would also read the embedded data and operate 
in accordance with the contents of that data; just as these 
methods would embed individualized data into the original 
software, so too would they retain this data in the illicit copy in 
order to identify its origin. 
[0003] 

[Problems Solved by the Invention] However, with methods 
such as those described above, a problem existed in that, since 
no data as all is written onto the portion which is displayed as 
video, it was possible to easily remove the embedded data by 
taking out only the displayed portion of the signal and replacing 
the newty required signal. An additional problem also existed in 
that, when recording to a format other than the format in which 
the original recording was performed - i.e., recording between a 
digital signal and an analog signal, or recording between a 
component signal and a composite signal, etc. - there was a 
good chance that any mforrnation not resting in the portion 
which is actually displayed would be dropped. 
[0004] An object of the present invention is to solve the 
aforementioned problems, providing a means and device for 
embedding data into a video signal such that the removal of the 
embedded data is difficult 
[0005] 

[Means of Solving the Problems] The present invention detects, 
as characteristic regions, elevated points or regions which 
sustain a value higher than a set value compared to the 
neighboring signal values, or sunken points or regions which 
sustain a value lower than a set value compared to the 
neighboring signal values, and embeds data in the video signal 
by either swapping out the aforementioned characteristic regions 
far another signal, or leaving them without any processing. 
Moreover, the present invention detects, as characteristic regions, 



elevated points or regions which rise beyond than a set value 
relative to the signal values on both sides in each line, or sunken 
points or regions tower than a set value relative to the signal 
values on both sides m each line, embeds data in the video 
signal by removing the aforementioned characteristic regions, 
and interpolates utilizing the neighboring signals, or leaves the 
signals without any processing. In addition, if consecutive 
characteristic regions are detected on the lines, and the 
characteristic regions form a single continuous region, then only 
the characteristic regions which are contained in those two- 
ended lines are detected as characteristic regions. Further, of the 
detected characteristic regions, a subset is selected at random, 
and the embedding data is embedded repeatedly. 
[0006] 

[Operation] With the present invention, based on the structure 
described above, the data to be embedded is embedded in the 
displayed video signal itself, and the location at which the data 
is embedded m the signal varies according to the image as well, 
so it is difficult to remove the embedded data without causing 
significant degradation in the video signal. Moreover, since the 
characteristic regions tn each line are detected and processed, it 
is easy to extract the embedded data as the signal is scanned in 
each line, and video degradation due to data embedding can be 
suppressed in each line so as to not be visually noticeable. 
[0007] In addition, if respective consecutive characteristic 
regions are detected on the lines, and the characteristic regions 
form a single continuous region, then only the characteristic 
regions which are contained in between the two ends of the lines 
are detected as characteristic regions, and in so doing, the 
characteristic regions can be made to become the edge portions 
of the objects being displayed, so if the original image is not 
known, then viewed in the direction of the Melds and frames it is 
difficult to find the regions at which degradation due to 
embedding has occurred. Further, of the detected characteristic 
regions, a subset is selected at random, and the data is embedded 
repeatedly in the selected characteristic regions, so even if an 
illicit copier detects the characteristic regions in the same 
manner as with the legitimate data embedding means and then 
attempts to embed nonsense data, it will be difficult to embed 
the same data repeatedly if the characteristic regions which were 
selected are not precisely known, and it is thus difficult for 
another party to embed data other than the data which was 
embedded using the legitimate embedding means. 
[0008] FIG. 1 shows a block diagram of die data embedding 
device in the first cmbodirnenL In FIG. 1, 1 is a characteristic 
region detection device which detects characteristic regions in 
the input signal; 2 is an interpolation processing device which 
generates a signal which replaces the signal in the characteristic 
regions by interpolating using the two adjoining pixels at the 
respective ends of the characteristic region; 3 is an error* 
correction encoding device which encodes the embedding data 
with error-correction codes; 4 is a temporary storage device 
which temporarily stores embedding data which has been 
encoded by the error-correction encoding device 3; and 5 is a 
selector which switches between either outputting the input 
signal as is as the output signal, or outputting the signal from the 
interpolation processing device 2. 
[Embodiments] 

[0009] The following is a description of the data embedding 
device of the present embodiment, which has the above structure. 
The input signal is a digital brightness video signal having a 
resolution of 8 bits per pixel, and is inputted to the characteristic 
region detection device 1 and the interpolation processing 
device 2, as well as to the selector 5. The characteristic region 
detection device 1 detects regions m the input signal which 
satisfy designated conditions, and sends the detection signal to 



the interpolation processing device 2 and the temporary storage 
device 4. 

[0010] FIG. 2 is a block diagram of the characteristic region 
detection device 1. In FIG. 2, 6 is a difference circuit which 
calculates the value of the difference with the immediately 
previous pixel; 7 is a delay element; 8 is a threshold circuit 
which outputs -1 if a specified level does not exceed -50, 0 if it 
is greater than -50 but less than 50, or 1 if it exceeds 50; 9 is a 
t em por ar y storage device which temporarily stores the signal 
which has been threshold processed; and 1 0 is a detection circuit 
which detects characteristic points based on the signal which has 
held in the temporary storage device 9. In terms of the behavior 
of the characteristic region detection device 1 having the above 
composition, after the difference between the input signal and 
the immediately previous input signal is calculated by the 
difference circuit 6, it is threshold processed by the threshold 
circuit 8 t and the locations which have varied by 50 levels or 
more are detected. This signal is held in the temporary storage 
device 9, and based on this signal the characteristic regions are 
detected by the detection circuit 10. 

[0011] FIG. 3 is a diagram which illustrates characteristic 
regions detected by the characteristic region detection device 1. 
Specifically, if an input signal is inputted in which, between 
adjoining pixels the intensity increases by 50 levels or more, 
then the variation between the next two pixels is less than 50 
levels, and then conversely the brightness of the fourth 
subsequent pixel decreases by more than 50 levels, or if an input 
signal is inputted in which, between adjoining pixels the 
intensity decreases by 50 levels or more, men the variation 
between the next two pixels is less than 50 levels, and then 
conversely the brightness of the fourth subsequent pixel 
increases by more than 50 levels, then pixels 1, 2, and 3 are 
detected as a characteristic region. 

[0012] Utilizing pixels 0 and 4, which are at the ends of the 
characteristic region, the interpolation processing device 2 
generates an interpolation signal for the three pixels in between, 
and sends it to the selector 5. FIG. 4 is an explanatory diagram 
of the processing in the interpolation processing device. 
Specifically, taking Y0 and Y4 to be the respective intensity 
values of pixel 0 and pixel 4, the pixel values Yl, Y2, and Y3 of 
the interpolation signal for pixels 1, 2, and 3 will be as shown in 
(value 1), (value 2), and (value 3) below. 
Yl=*/«Y0 + '/iY4 (valuel) 
Y1 = 14Y0 + I AY4 (valuel) 
Yl« , /*Y0 + , /«Y4 (valuel) 

Meanwhile, the embedding data is error-correction encoded by 
the error-correction encoding device 3, and stored in the 
te m porary storage device 4. In the temporary storage device 4, 
each time the detection signal is inputted from the characteristic 
region detection device 1 , embedding data is outputted one bit at 
a time to the selector's controller. However, when the 
embedding data is outputted sequentially one bit at a time, and 
output of all of the bits has ended, the embedding data is 
repeatedly outputted sequentially once again, starting from the 
first bit In addition, the data embedding location or the starting 
location is randomly selected, and some of the detection regions 
are ignored, such that the embedding data is embedded about 
once every few seconds starting from the detected characteristic 
region* The selector 5 normally outputs the input signal as is, 
and selects the signal from the interpolation processing device 2 
as the output only if the signal from the temporary storage 
device 4 is logical V. 

[001 3] The characteristic region which is detected by the present 
embodiment as described above is merely a portion of the video 
signal, and the degradation due to data embedding is only in one 
line so, visually, data can be embedded without being noticeable. 



In addition, the interpolated signal which replaces the 
characteristic region utilizes the signal at the ends of the 
characteristic region and takes on the weighted average thereof, 
and thus, when viewed on the one line, takes on values which 
vary continuously from the signal values in the vicinity. It is 
therefore difficult to infer the locations at which data has been 
embedded from merely observing the signal an the line, and it is 
necessary to infer these using two-dimensional data such as the 
correlation with other lines, so it is difficult to infer just the 
embedded portions. 

[0014] In addition, die embedding data is error-correction 
encoded, and is repeatedly embedded into the video signal, so 
there will not be any situations in which the embedded data 
cannot be read due to degradation from copying. 
[001 5] m the present embodiment, regions such as those shown 
in FIG. 3 are detected, but moreover the size of the brightness 
variations or the size of the regions can also be changed, and it 
is also possible to select characteristic points which differ in 
quality. In this manner, illicit copiers are prevented from writing 
T data into all regions which may possibly be detected as 
characteristic regions. This is due to the feet that, if detection 
and data embedding are performed while continually changing 
conditions in the software such as characteristic region size and 
brightness variation, then for an illicit copier who does not know 
what changes were made, die number of regions which might be 
detected as characteristic regions becomes astronomical, and the 
removal of all of the characteristic regions without causing 
significant degradation can thus be made very difficult 
[0016] Furthermore, a digital brightness signal was utilized as 
the input signal, but this can be performed on individual signals 
for the respective component colors as well, and analog 
component and composite signals can also be converted into 
digital signals and input, and men upon output can be converted 
back. In addition, the portions in which characteristic regions 
have been extracted were interpolated utilizing the respective 
single pixels adjacent to the ends, but multiple adjacent pixels 
can also be used, or another method can be used if the removed 
region proves difficult to find. Moreover, in the present 
embodiment, the embedding data was inputted to the error- 
correction encoding device 3, but can also be inputted after 
being encrypted first, and a signal can also be inputted which 
has been encrypted with a different value every time it was 
repeatedly embedded. 

[0017] FIG. 5 is an explanatory diagram illustrating the 
characteristic [region] detection method of the data embedding 
device in the second embodiment of the present invention. The 
characteristic regions on the respective lines have a portion in 
which the characteristic regions on adjacent lines overlap at a 
position in the horizontal direction, and these characteristic 
regions represent die case in which a single continuous 
characteristic region is formed. The structure of the present 
embodiment is the same as die structure in FIG. 1 , differing only 
in the processing in the characteristic region [detection] device 1. 
The characteristic region [detection] device 1 temporarily stores 
one frame's worth of video, and detects characteristic regions as 
described in FIG. 2; however, it detects regions in which the 
brightness changes significantly at the ends, and assumes that 
there is no change in the size of the characteristic region. The 
position of die characteristic region which is detected next is 
compared from line to line and, as shown in FIG. 5, if these 



regions have a position in the horizontal direction at which they 
overlap between adjacent lines, then they are treated as though 
they were generated by the same object which is displayed in the 
video, they are detected as a [single] region which was formed 
by the combination of these, and among these the region on the 
highest tine and the region on the lowest line are detected as 
characteristic regions which will be candidates for data 
embedding. However, when the highest pine] and the lowest 
[line] represent the top and bottom of the screen, they will not be 
detected. For characteristic regions detected in this manner, the 
same processing is performed as for the first embodiment. 
[0018] Based on the above structure, die present embodiment 
will detect as characteristic regions the edge portions of objects 
shown in a video, so provided that the original signal is not 
known, even if the characteristic regions are taken out it is 
difficult to ascertain whether video which was originally filmed 
has been taken out or if it was not present m the first place, and 
it is difficult to discover the embedding positions by, for 
instance, merely calculating the correlation between lines. 
[0019] m addition, the present embodiment detected, as data 
embedding position candidates, the top and bottom portions of 
the region created from consecutively detected characteristic 
regions, but inferring the embedding position by calculating the 
correlation between the lines can also be made difficult by 
detecting as an embedding position candidate die region on the 
line containing the region which is smallest among the regions 
on the respective lines. 
[0020] 

[Effects of the Invention] As described above, based on the 
present invention, data can be embedded into a video signal 
without causing significant visual degradation to the video, and 
in addition, discovering the locations of this embedding is 
difficult, and data embedding methods and devices can be 
provided in which the embedded data is not susceptible to 
removal, overwriting, and the like, which has significant 
practical effects. 

[Brief Description of the Drawings] 

[FIG. 1 ] Block diagram of the data embedding device in die first 
embodiment of die present invention 

[FIG. 2] Block diagram of the characteristic region detection 
device in this embodiment 

[FIG. 3] Pixel state diagram of a characteristic region detected in 
this embodiment 

[FIG. 4] State diagram showing the processing in the 

interpolation processing device in this embodiment 

[FIG. 5] State diagram of a characteristic region delected in the 

second embodiment of the present invention 

[FIG. 6] Pattern diagram showing a conventional data 

embedding method 

[Key] 

1 Characteristic region detection circuit 

2 Interpolation processing device 

3 Error-correction encoding device 

4 Temporary storage device 

5 Selector 

6 Difference circuit 

7 Delay element 

8 Threshold circuit 

9 Temporary storage device 

10 Detection circuit 
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[ooio]02li, ftftttttSttiSIH a«tftH-?fe 
«. B2l=j3t%-c. 61*. 10W©B*£ 
*f*£#lalB. 7 1*. as**. 8 1*. BfSU^;u4< 
— 5 OsfeSSJ-C— 1 £. — 5 0£l±5 OKlTTfOS. 50 
$fflx*A*-ei Stt*-r.*L*LxfllElB, 91*. L£ 

20 ^ffiflia**ifc«^^-SE«-i-*-^E«8«. 1 o 

I*. -&Ett8«9(=*A&;h.fc«*£t£l::»ttfi£ 
ftflKfSttHJlHlB-efcS. &±a>J:5l::****ifc»« 

I*. S#BB6T1o|&©«*fc©S#A<mt£*ifc 
*. L#l\ffl@B8^L*L^fflS**l,. 5 0L"<;Uy. 

«9l=Sii.b4i. *©«*£tfcl::»fflffli*j!i<«aila]B 
1 O-Pftffl*^. 

[ooi i] S3 1*. ftatttt&ttiss 1 -cstta-r *>ft 
o i/X/ny±SSj!)<*in L . **u=tt < 2 HSJRW sou 
(=5 0 ls*/l,VL±8aL&X'p+& J: 5 fc«**<A* S*t 

fc £ t . *fci*»SE*M-e s o U"gu]a±as*<x4> 

< . *0)X(OA.Wmtt>0>{t*tfmz 5 0 UK,Ujy±»£ 
A<ltftl1-*J:5)S«^A***tfc£t. ®*1,2,3 

[0 0 12] fflWSHSg»2rii. ^«€a*CD^4«C0K 
40 *0. 4$fflLNrl«llzfe*3IHII<0»|HHt*S*«L. 

-t \s<? z 5 i=a« . m 4 1*. **iHifflsa« 2 -coasco 
K^a-cfc* 0 -r^*,. n*o,s*4ro»aa$-* 

41-^tlYO. Y 4t LT. 11*1. 2, 3<0ttlBfl^OlI 
*filY 1. Y2. Y3I*. BITO (»1) (ft 2) (ft 
3) «>J:3I=4*. 

Y 1 =3/4Y0+1/4Y4 (ft 1 ) 
Y2=1/ZY0+1/2Y4 (»2) 
Y3= 1/4Y0+3/4Y4 (ft3) 

-a. afta^it«i*. KyiriE»^t8ii3-T?RyiT 

50 IE»^ft**i. -^E«K«4lz|E«*ti*. -^E« 
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BS*!>a**.<&8flE<D»i::fc*«fc3i=. it«s»a*t& 

[0 0 13] J3l±©<fc5l=**Jfifflt«k*ilftttb*^4 ' 
INMMfi. tt««*+©«-ffl-ea&y, flMSStta* 

ts«««ro^*5yic«mi-r-s«^i±, ««Ma«©ig«s© 
i^-rvfc-cjifra^ ia«<D«*^bit«iwt«<bL 

£«fca^*M£*£*Sc£l*HliT!&y. ffi©^ 
[0014] * 61=. Mtb&*tim\t. HyiDEtt^b 

[0015] fcfc, *Wtfl-eii. B3iz^-r.fc5«ffi 
*SttUlLfcA<. ^0«<b©*§?*££;LTfc«g*a> 30 

fcfty. *£4£<b£#?ci:fc<*-^T<Dft«ffli*£ 
[0 0 16] A^fl^^LT. t<5>*/WMME 

2* A0fl»fc-jJU«LTA»U a* 

**«ta»i::*ifti,iii*ii«fco. ^e«i«$%y 

WLfc*. mil=Mtt-r«4iSa>m«JBl\T4iJ:l% 

u *fc» ay»^*tfcfi«$36ftur e><-f4t<7)7? 

tt. aaa*l*«£RyinE»^b«1l3^AaU= 50 



Jt><* 5fcl=Bt<34bLT-tf>ofcfiL AALT*?TtJ:^ 
L. 2&f=il!*3l«ftafcftl=ll&?feai=«*<b* 

[0 0 17]BSIt ttM0ft2aj|IMl=&ft«tt 
«a»a*8K©«*tta»ai=OOT©IMI|B|-C. 2 

1 ©ana©*** y. 

■ 11*1 7U-A#<0tt«£-B#MI::E1tU B2T?» 
MLfcWMW«ttlii^«. fcf£U »««*©*#* 

«£«hits *,©£•*-$„ kiukiuL&miiMxn 

SIrI ©tt«A<Bf » 7 <f Vflfl T?M« 4, o 4 t> li, 

**it>tt, tt«*(=ttLtt*4ifcBI-<D*(*l=«fc-3T± 

<t LT«ffi-r41#15MH«<t LT. U(W7-f>±i:* 
*ffl*£«Ttt©7<>±l::fcSffii* LT 

&mf-&o teu m±Bm.Tmmma>WL±&*.TB 

fc«MMtl=oni\ *l©3SI6«.fcRI«©fllil£fT7. 
[0 0 18] J3Lt©«jSf=,fcy. *Stt«-Cfli. K*f= 

fcft. ,*«««^yifcMi.fc»£T*4,. :*-'Jv-*-;U© 

«#Sfl&ftL>fBy. jc-<rttLai**irufc*a)^aiy 

Bf-fSctttHK-e&y. J»lc. 7-r>IH©ftH£ft* 

[ooi9] tats* *sete«-ei±. a»LT«as*ifc 

[0020] 

a^^Wlctt«!ro35ft^l^i^t*#^^ttL^■^?tt«fl|#|*)l= 
itfc't c t immvib y . g*>a*fli«£B y Ht^^ifc 

[BSOffiJU^K^] 

[si 1 ] *£Ham 1 rosejs«ic*3it*flte«<6a^K 
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[B3] BX£ffl::£^Ttttfld:K«4MMnft0B*tt 



CH4] nftftfli=&it««nawwfl<0A9£ai-*~ 

ttBB 

[■5] *fin*2(D|lKffll:£^Ttttt>$^«tt«fl 
ttttttBB 

[■6] «IXOllHll«0^a«»t/^->B 
1 W««*t*liJEtt 



2 tm«Hti 

3 HyiTZIHHUHl 

4 -mens* 

s -tru** 

6 S#[E]B 

7 SHIT 

8 L*l^«BB 

9 -KE1KI 
i o tttbBft 
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